1. Statement of the Technical Field
The inventive arrangements relate to teleoperation systems and methods for the electronic remote control of unmanned machines. More particularly, the invention concerns teleoperation systems and methods for smoothing unpredictable latencies (or time delays) experienced in communications links (or data links) between controllers and remote unmanned machines.
2. Description of the Related Art
Teleoperation systems typically comprise hand controller systems for remotely controlling movement of an unmanned slave device via a wired or wireless communications link (or data link). In some scenarios, the remote control of the slave device is an Internet based control. In all cases, the slave device can include, but is not limited to, a robot, a robotic manipulator, an Unmanned Aerial Vehicle (“UAV”), an Unmanned Ground Vehicle (“UGV”), and/or an Unmanned Underwater Vehicle (“UUV”). During operation, a hand controller system (sometimes in the form of a joy stick or a pistol grip) senses the movement by a user's hand. Thereafter, the hand controller system generates one or more control messages. The control messages are sent over a wired or wireless data link to the slave device. At the slave device, the control messages are used for actuating various motion operations of the slave device. The control system of the slave device will respond to these control messages by activating certain servo-motors, solenoids, or other devices within the slave device to produce the desired action. Video cameras can be used to provide visual information to a remote operator regarding a work piece and/or the environment surrounding the slave device. Haptic technology can also be employed to provide tactile feedback by the teleoperation system to a user. The tactile feedback is typically provided by applying forces, vibrations and/or motions to one or more portions of the hand controller.
Notably, unpredictable inherent latency can be experienced in the data links between the hand control system and the slave device. For example, the slave device is a UGV with a radio. The UGV drives over a hill, whereby the remotely located human operator controlling the UGV loses Line Of Sight (“LOS”) thereof. Also, the quality of communications link degrades. In this case, the inherent latency continuously varies unpredictably within, for example, a range of ten milliseconds to hundreds of milliseconds. The operator controlling the UGV is aware of the inherent latency between the time a control message is sent from the hand controller system and the time the control message is received by slave device. Consequently, the operator tries to mentally compensate for the inherent latency. In this regard, the operator tries to compensate for a relatively low inherent latency when the inherent latency is actually a relatively high inherent latency, or vice versa. In effect, the operator commands a movement of the slave device that is incorrect because the slave device is actually in a different location than that thought by the operator. As a result, the slave device may undershoot or overshoot a point or a target, thereby making the teleoperation system unpredictable. In the UAV scenarios, the unpredictable natural latency leads to unstable control of the UAV by the operator. The unstable control may cause the UAV to crash. One can appreciate that such unpredictable and unstable control of slave devices is undesirable, especially in a military and rescue contexts.
Many conventional systems do not implement any solutions to the variable inherent latency problem. In these systems, teleoperation commands are executed when they are received by the slave device. Also, video feedback is shown to the operator as it is received. Other conventional systems employ stabilization algorithms to address the variable inherent latency problem. In these systems, the stabilization algorithms prevent the systems from becoming dangerously unstable. For example, a system comprises a remotely controlled UAV. If the operator is providing incorrect commands as a result of the variable inherent latency, then a control system on the UAV rejects or modifies the commands received thereat so to prevent crashing of the UAV. Despite the advantages of the stabilization algorithms, they do not address the issue of predictability of the teleoperation systems.